Introduction¶

In this notebook we will explore the extracted features from the WESAD dataset.

%reload_ext pretty_jupyter

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn import metrics

import sklearn.feature_selection as fs

import seaborn as sns
import plotly_express as px
import plotly.offline as pyo
import matplotlib.pyplot as plt

pd.set_option('display.max_columns', 100)
pd.set_option('display.max_rows', 100)
pyo.init_notebook_mode()

General Analysis¶

.tabset|O_O|.tabset-pills|O_O|.tabset-fade

First, we import the dataset.

data = pd.read_csv('../data/03_primary/WESAD/combined_subjects.csv')

We can observe that the all the data is numeric and there are no missing values. We will remove the first column as it is just a clone of the index.

data = data.drop([data.columns[0]], axis=1)

Dataset Column Overview¶

data.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 2091 entries, 0 to 2090
Data columns (total 44 columns):
 #   Column           Non-Null Count  Dtype  
---  ------           --------------  -----  
 0   net_acc_mean     2091 non-null   float64
 1   net_acc_std      2091 non-null   float64
 2   net_acc_min      2091 non-null   float64
 3   net_acc_max      2091 non-null   float64
 4   EDA_phasic_mean  2091 non-null   float64
 5   EDA_phasic_std   2091 non-null   float64
 6   EDA_phasic_min   2091 non-null   float64
 7   EDA_phasic_max   2091 non-null   float64
 8   EDA_smna_mean    2091 non-null   float64
 9   EDA_smna_std     2091 non-null   float64
 10  EDA_smna_min     2091 non-null   float64
 11  EDA_smna_max     2091 non-null   float64
 12  EDA_tonic_mean   2091 non-null   float64
 13  EDA_tonic_std    2091 non-null   float64
 14  EDA_tonic_min    2091 non-null   float64
 15  EDA_tonic_max    2091 non-null   float64
 16  BVP_mean         2091 non-null   float64
 17  BVP_std          2091 non-null   float64
 18  BVP_min          2091 non-null   float64
 19  BVP_max          2091 non-null   float64
 20  TEMP_mean        2091 non-null   float64
 21  TEMP_std         2091 non-null   float64
 22  TEMP_min         2091 non-null   float64
 23  TEMP_max         2091 non-null   float64
 24  ACC_x_mean       2091 non-null   float64
 25  ACC_x_std        2091 non-null   float64
 26  ACC_x_min        2091 non-null   float64
 27  ACC_x_max        2091 non-null   float64
 28  ACC_y_mean       2091 non-null   float64
 29  ACC_y_std        2091 non-null   float64
 30  ACC_y_min        2091 non-null   float64
 31  ACC_y_max        2091 non-null   float64
 32  ACC_z_mean       2091 non-null   float64
 33  ACC_z_std        2091 non-null   float64
 34  ACC_z_min        2091 non-null   float64
 35  ACC_z_max        2091 non-null   float64
 36  0_mean           2091 non-null   float64
 37  0_std            2091 non-null   float64
 38  0_min            2091 non-null   float64
 39  0_max            2091 non-null   float64
 40  BVP_peak_freq    2091 non-null   float64
 41  TEMP_slope       2091 non-null   float64
 42  subject          2091 non-null   int64  
 43  label            2091 non-null   int64  
dtypes: float64(42), int64(2)
memory usage: 718.9 KB

Dataset Shape¶

Descriptive Statistics¶

Now we will explore the data. We will start by looking at the distribution of the features.

Feature selection¶

.tabset|O_O|.tabset-pills|O_O|.tabset-fade

After loading and observing the dataset, it's time to find best features.

# def get_best_features()
kBest = fs.SelectKBest(fs.f_classif, k=5)
res = kBest.fit_transform(data.drop(columns=['label']), data['label'])
filter = kBest.get_support()
df = pd.DataFrame(res, columns = data.columns[:-1][filter])
df = df.join(data['label'])
print(f"""Top features:\n{"  ".join(data.columns[:-1][filter])}""")

Top features:
net_acc_std  net_acc_max  EDA_tonic_mean  EDA_tonic_min  EDA_tonic_max

cdf = pd.concat([df.drop("label", axis=1), pd.get_dummies(df["label"])], axis=1)
cdf.rename(columns={0: "amusement", 1: "baseline", 2: "stress"}, inplace=True)

corr = cdf.corr()
fig = px.imshow(corr[["amusement", "baseline", "stress"]], text_auto=True)
fig = fig.update_layout(width=500, height=800)
fig.show()

Data analysis by subject¶

.tabset|O_O|.tabset-pills|O_O|.tabset-fade

from plotly.subplots import make_subplots
import plotly.graph_objects as go

def plot_distribution(feature, nbr_cols=4):
    subjects = [2, 3, 4, 5, 6 ,7 ,8, 9, 10, 11, 13, 14, 15, 16, 17]
    titles = [f'Subject {x}' for x in subjects]
    plot = make_subplots(rows=len(subjects) // nbr_cols+1, cols=nbr_cols, subplot_titles=titles)

    row_n = 1
    col_n = 1
    for sub in subjects:
        csv = pd.read_csv(f'../data/03_primary/WESAD/subject_feats/S{sub}_feats_4.csv')
        plot.add_trace(go.Bar(y=csv[feature]), row_n, col_n, )
        col_n += 1
        if col_n > nbr_cols:
            col_n = 1
            row_n += 1

    plot.update_layout(height=1000)
    plot.show()

net_acc_std¶

plot_distribution('net_acc_std')

net_acc_max¶

plot_distribution('net_acc_max')

EDA_tonic_mean¶

plot_distribution('EDA_tonic_mean')

EDA_tonic_min¶

plot_distribution('EDA_tonic_min')

EDA_tonic_max¶

plot_distribution('EDA_tonic_max')

The effect of the stress level¶

.tabset|O_O|.tabset-pills|O_O|.tabset-fade

def get_label(label):
    frame = df.loc[df.label==label]
    frame.index =  range(0, frame.shape[0])
    frame.index = pd.to_datetime(frame.index, unit='s')
    return frame

amusement = get_label(0)
baseline = get_label(1)
stress = get_label(2)

df.label = df.label.replace(0, 'amusement')
df.label = df.label.replace(1, 'baseline')
df.label = df.label.replace(2, 'stress')
df

net_acc_std¶

fig = px.line(df, y='net_acc_std', color='label')
fig.show()

net_acc_max¶

fig = px.line(df, y='net_acc_max', color='label')
fig.show()

EDA_tonic_mean¶

fig = px.line(df, y='EDA_tonic_mean', color='label')
fig.show()

EDA_tonic_min¶

fig = px.line(df, y='EDA_tonic_min', color='label')
fig.show()

EDA_tonic_max¶

fig = px.line(df, y='EDA_tonic_max', color='label')
fig.show()